It has widely been performed to convert an image captured with an in-vehicle camera in such a manner as if the image were captured in a virtual line of sight (e.g., a line of sight looking down on a vehicle from immediately above the vehicle) different from the line of sight of the in-vehicle camera and display the resulting line-of-sight-converted image on an in-vehicle monitor. This line-of-sight-converted image is generated as follows. First, when the direction of the line of sight when the image is actually captured and the direction of the virtual line of sight are determined, pixels in the actually captured image and pixels in the line-of-sight-converted image correspond to each other on a one-to-one basis. Thus, the image data (such as luminance or chroma) of the pixel in the captured image corresponding to each single pixel in the line-of-sight-converted image is set. Accordingly, the line-of-sight-converted image can be generated based on the captured image.
When it is assumed herein that, e.g., the size of a line-of-sight-converted image is 800 pixels×500 pixels, the number of the pixels is as large as 400,000. It is difficult to display a line-of-sight-converted image in a imaging period (e.g., 30 Hz) by calculating the corresponding pixel in the captured image for each of the 400,000 pixels. Accordingly, a method which preliminarily produces a conversion table showing a correspondence relationship specifying which pixel in the captured image is to be allocated to which pixel in the line-of-sight-converted image has been used commonly. When the preliminarily produced conversion table is stored to be readable, the corresponding pixels in the captured image can be specified by merely referencing the conversion table, allowing prompt application of the line-of-sight-conversion to the captured image.
However, when the mounting position or mounting angle of the in-vehicle camera with reference to a road surface changes, the stored conversion table can no longer be used. For example, when a vehicle is running on a curved road, is accelerated or decelerated, or has a heavy object mounted therein, the vehicle is inclined so that the mounting position or mounting angle of the in-vehicle camera with reference to a road surface changes. The mounting position or mounting angle of the in-vehicle camera relative to the vehicle may also change due to the loosening of the fastened portion of the in-vehicle camera or the like. After the mounting position or mounting angle of the in-vehicle camera with reference to a road surface thus changes, when the stored conversion table is used to perform the line-of-sight conversion, the resulting line-of-sight-converted image looks artificial.
Accordingly, a technology has been proposed to update the conversion table in accordance with a new mounting position and a new mounting angle when the mounting position and mounting angle of the camera with reference to a road surface change (Patent Literature 1).